A Linear Programming Approach to Nonstationary Infinite-Horizon Markov Decision Processes
نویسندگان
چکیده
منابع مشابه
A Linear Programming Approach to Nonstationary Infinite-Horizon Markov Decision Processes
Nonstationary infinite-horizon Markov decision processes (MDPs) generalize the most well-studied class of sequential decision models in operations research, namely, that of stationaryMDPs, by relaxing the restrictive assumption that problem data do not change over time. Linearprogramming (LP) has been very successful in obtaining structural insights and devising solutionmeth...
متن کاملA linear programming approach to constrained nonstationary infinite-horizon Markov decision processes
Constrained Markov decision processes (MDPs) are MDPs optimizing an objective function while satisfying additional constraints. We study a class of infinite-horizon constrained MDPs with nonstationary problem data, finite state space, and discounted cost criterion. This problem can equivalently be formulated as a countably infinite linear program (CILP), i.e., a linear program (LP) with a count...
متن کاملExtreme point characterization of constrained nonstationary infinite-horizon Markov decision processes with finite state space
We study infinite-horizon nonstationary Markov decision processes with discounted cost criterion, finite state space, and side constraints. This problem can equivalently be formulated as a countably infinite linear program (CILP), a linear program with countably infinite number of variables and constraints. We provide a complete algebraic characterization of extreme points of the CILP formulati...
متن کاملAverage Optimality in Nonhomogeneous Infinite Horizon Markov Decision Processes
We consider a nonhomogeneous stochastic infinite horizon optimization problem whose objective is to minimize the overall average cost per-period of an infinite sequence of actions (average optimality). Optimal solutions to such problems will in general be non-stationary. Moreover, a solution which initially makes poor decisions, and then selects wisely thereafter, can be average optimal. Howeve...
متن کاملInformation Relaxation Bounds for Infinite Horizon Markov Decision Processes
We consider the information relaxation approach for calculating performance bounds for stochastic dynamic programs (DPs), following Brown, Smith, and Sun (2010). This approach generates performance bounds by solving problems with relaxed nonanticipativity constraints and a penalty that punishes violations of these constraints. In this paper, we study infinite horizon DPs with discounted costs a...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Operations Research
سال: 2013
ISSN: 0030-364X,1526-5463
DOI: 10.1287/opre.1120.1121